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SUMMARY 


This article describes the International Monetary Fund's (IMF) Data Quality Assessment 
Framework (DQAF) and broadly assesses the effectiveness of Australia's national accounts 
statistical program against the DQAF's five dimensions of quality. Overall, Australia's national 
accounts perform well in all areas of the DQAF with particular strength in the areas of integrity, 
methodological soundness, serviceability and accuracy and reliability. In the other area of quality, 
accessibility, the national accounts perform well but improvements can be made as user 
requirements continue to change. 


The ABS is planning to do a comprehensive assessment against the IMF's DQAF for the national 
accounts. The assessment will be publicly available and is likely to be completed by early 2004. 


The development of an ABS data quality framework and associated quality measures for its 
economic accounts is also planned. In this work the ABS will draw upon the IMF's DQAF, but will 
also modify and extend aspects of quality to suit local circumstances and user needs. The 
proposed ABS quality framework and its application to the national accounts statistical program 
will be the subject of a future article. 


BACKGROUND 


Work on an approach to assessing data quality was begun in 1997 by the Statistics Department 
of the IMF. Building on this earlier work, in 2001 the IMF developed its DQAF for assessing the 
quality of data used for macroeconomic analysis. The DQAF is an integrated and flexible 
framework that serves as an umbrella for dataset-specific frameworks, including national 
accounts. The System of National Accounts 1993 (SNAQ3) is used as the benchmark for 
definitions, concepts, classifications and valuation methods. Data quality is assessed using a six 
part structure that spans institutional structures, statistical processes and characteristics of the 
statistical products, recognising the need for trade-offs among the various aspects of data quality. 


Three main uses were envisaged in developing the IMF's DQAF. First, as a tool for guiding 
countries’ efforts to strengthen their statistical systems though the provision of a means of self- 
assessment which would facilitate the identification of areas for improvement. Second, as a guide 
for IMF staff in assessing the quality of data provided by member countries and in designing 
programs of technical assistance and third, to assist data users to assess data quality for their 
own purposes. 


The DQAF has not been designed to be a checklist or audit of statistical practices as it does not 


cover exhaustively all issues at the detailed level. Further, in developing its assessment 
framework, the IMF explicitly recognised the need for member countries to take into account their 
own particular circumstances. The IMF expects that the frameworks will be applied flexibly with 
the objective of pointing to relevant areas that may need attention so that an action plan and 
resource requirements can be identified. 


The IMF's work on quality frameworks has the strong support of the ABS. The ABS regards 
quality assessments of its statistics and methods as extremely important, both to determine 
areas that are deficient so that improvements can be made, and to assist users in their 
understanding of the statistics so that they can use them more effectively. As the ABS generally 
complies with the SNA93 benchmark standards underlying the DQAF, the IMF quality framework 
provides a good foundation on which to build a comprehensive quality assessment system for 
Australia's national accounts. 


THE IMF DATA QUALITY ASSESSMENT FRAMEWORK 


The IMF DQAF covers five dimensions of quality and a set of prerequisites for the assessment of 
data quality. The framework recognises that data quality is intrinsically bound with that of the 
institution producing it as well as the ‘quality of the individual data product’. Thus the DQAF also 
includes some elements and indicators which, while not constituting a quality dimension, are 
preconditions for quality. They generally refer to desirable attributes of the agencies operating the 
statistical system. The following briefly describes the set of prerequisites and the statistical 
practices associated with each of the five dimensions of quality. 


e Prerequisites of quality—the environment is supportive of statistics, resources are 
commensurate with the needs of statistical programs, and quality is a cornerstone of 
statistical work. 


Integrity—statistical policies and practices are guided by professional principles, statistical 
policies and practices are transparent, and policies and practices are guided by ethical 
standards. 


Methodological soundness—concepts and definitions used are in accord with 
internationally accepted statistical frameworks; the scope is in accord with internationally 
accepted standards, guidelines, or good practices; classification and sectorisation systems 
are in accord with internationally accepted standards, guidelines, or good practices; and 
flows and stocks are valued and recorded according to internationally accepted standards, 
guidelines, or good practices. 


Accuracy and reliability—source data available provide an adequate basis to compile 
Statistics; statistical techniques employed conform with sound statistical procedures; source 
data are regularly assessed and validated; intermediate results and statistical outputs are 
regularly assessed and validated; and revisions, as a gauge of reliability, are tracked and 
mined for the information they may provide. 


Serviceability—statistics cover relevant information on the subject field; timeliness and 
periodicity follow internationally accepted dissemination standards; statistics are consistent 
within the data set, over time, and with other major data sets; and data revisions follow a 
regular and publicised procedure. 


Accessibility—statistics are presented in a clear and understandable manner, forms of 
dissemination are adequate, and statistics are made available on an impartial basis; up-to- 
date and pertinent metadata are made available; and a prompt and knowledgeable support 
service is available. 


ASSESSMENT OF THE ABS NATIONAL ACCOUNTS PROGRAM AGAINST THE IMF 
FRAMEWORK 


Given the importance of the national accounts for economic policy formulation and for monitoring 
the impact of those policies, it is important that they are of a high quality. The quality of a 
particular statistic is determined by its ‘fitness' for whatever purposes users have for the statistics. 


Because the compilation of the national accounts is a complex task involving the use of many 
diverse data sources and methods, it is not possible to provide a single, comprehensive measure 
of the quality of the estimates. Nonetheless, it is possible to gain an insight into their quality by 
analysing each aspect of quality separately and then bringing them together to obtain an overall 
picture. 


In understanding quality, it is also important to recognise that there are often trade-offs across the 
various aspects of quality. For example, statistics that are very timely are often less accurate than 
those that are less timely. Trade-offs can also occur between accuracy and relevance. It is also 
important to recognise that, in general, the higher the quality of an estimate, the more costly it is 
to compile it. Ultimately the assessment of whether data are of high quality, i.e. whether they are 
‘fit for purpose’, must lie with the user. 


The ABS considers that it observes the statistical practices associated with each of the IMF's 
quality dimensions. A brief discussion of each dimension follows. The ‘prerequisites of quality’ 
and the ‘integrity’ dimension are only touched on very briefly in this article as they are essentially 
corporate in nature. Users who would like further information regarding either of these 
dimensions from an ABS wide perspective should refer to the ABS Annual Report and the ABS 
Corporate Plan, both of which are available on the ABS web site. 


Prerequisites of quality 


The legislative framework, organisational structure, planning and quality management practices 
of a statistical agency all affect the ability of a statistical program to meet user needs efficiently 
and effectively. The ABS operates within the framework of a basic statistics act and an additional 
act relating specifically to its statistical program. The principal legislation determining the 
functions and responsibilities of the ABS are the Australian Bureau of Statistics Act 1975 and 
the Census and Statistics Act 1905. 


The ABS is headed by the Australian Statistician—a statutory office. The Australian Statistician 
determines which statistics are collected, after full discussion with users and the Australian 
Statistics Advisory Council, and makes the results widely available. The independent status of 
the Australian Statistician is specified in law. This helps to ensure the ABS's impartiality and 
independence from political influence. 


The Census and Statistics Act 1905 provides the Statistician with the authority to conduct 
statistical collections and, when necessary, to direct a person to provide statistical information. 
The same Act imposes on the ABS obligations to publish and disseminate compilations and 
analyses of statistical information and to maintain the confidentiality of information collected 
under the Act. 


The Australian Bureau of Statistics Act 1975 requires the Statistician to prepare an annual 
report on the operations of the ABS for presentation to Parliament. This Act also established the 
Australian Statistical Advisory Council to assist the ABS fulfil its role. The Council is the key 
advisory body to the ABS and provides valuable input to the directions and priorities of the ABS 
work program. The ABS is financed via Parliamentary appropriations for its administration and 
programs. Its financial statements are audited annually by the Australian National Audit Office. 
Together these elements provide an environment conducive to the production of high quality 


national accounts and other statistics. 


While working within this sound and established framework, the ABS regularly reviews its 
organisation and structure to ensure that its operations remain efficient, that resources are 
focussed on providing ‘statistical leadership’ and that its program remains well placed to take 
advantage of emerging opportunities, including making the best use of all available data. 


Integrity 


This dimension of quality relates to the need for statistical systems to be based on the principles 
of objectivity in the collection, compilation and dissemination of data to ensure unbiased statistics 
which are not subject to confidentiality breaches or premature release. The dimension covers 
those aspects of the statistical program that relate to policy, transparency and ethical standards. 


The ABS has an excellent reputation, both at home and abroad, for providing a statistical service 
of quality and integrity. This, in particular, is recognised by those who influence opinion such as 
parliamentarians, government agency heads, media commentators and the international 
statistical community. A key reason for this reputation is that the ABS is willing to subject its 
operations and performance to both internal and external scrutiny. Some of the ways that this is 
done include: 


e The advertisement of all scheduled release dates for publications up to twelve months in 
advance. Daily press and media releases inform users of publications being released each 
day. This information is also available on the ABS web site. Release of all publications is 
subject to a strict publicly known embargo policy that ensures impartiality, an essential 
element of integrity. 


e The publication of the ABS three year Forward Work Program demonstrates the open 
statistical process by describing for each ABS program the resources, outputs, clients and 
uses of statistical information, and the proposed main developments over the next three 
years. The Forward Work Program is published in hard copy each year and is also 
available on the ABS web site. 


The release of information about statistical standards, frameworks, concepts, sources and 
methods in a range of information papers and other publications in hard copy or 
electronically to ensure that the public are informed about changes to statistical processes. 
Details of major revisions to published data are described in the explanatory notes of the 
relevant publication. 


Methodological soundness 


This dimension of quality covers the need for the methodological basis for the production of 
statistics to be firmly based on international standards, guidelines and generally accepted 
practices. Within this dimension the DQAF highlights five aspects of quality namely, frameworks, 
concepts and definitions, scope and coverage, sectorisation and classifications. Overall, the 
Australian national accounts cover these five aspects well. 


The Australian system of national accounts (ASNA) is based on the international standard, the 
SNAQ3. Aside from the core national accounts of GDP and income, the ASNA includes both 
national and sectoral financial accounts, capital accounts, and balance sheets. The ASNA also 
encompasses input-output tables, multifactor productivity statistics, and accounts for each state 
and territory. Many Australian national accounts estimates are also disaggregated to provide 
information about economic assets and activities for sectors, industries, and commodities, and 
about different types of assets, liabilities, transactions and other economic events. The ultimate 


scope of the ASNA encompasses the full range of statistics that SNA93 recommends for a 
complete national accounting system. However, at this stage of its development, the ASNA does 
not include all of the elements of the SNA framework, although Australia's implementation of 
SNAQ3 is extensive relative to most other countries. The ABS is continuing to work towards 
further improving and extending the range of its statistics. 


The concepts and definitions used in the ASNA generally conform with those set out in SNA93 
but some minor variations have been adopted to allow for particular Australian data supply 
conditions or user requirements. Such variations are relatively minor and are noted at appropriate 
points in the Australian System of National Accounts, Concepts, Sources and Methods (cat. 
no. 5216.0). 


Standard classifications and definitions of statistical units and items are essential elements 
underlying the compilation and presentation of statistics produced by national statistical offices. 
The use of such standards ensures that statistics are comparable across industry and sector 
boundaries and can be aggregated from various collections. The ABS has a range of standard 
economic classifications, including those for institutional sector, industry and commodity. These 
classifications are used in the national accounts and in other ABS statistics and, occasionally, in 
non-ABS statistics. The ABS places great emphasis on ensuring that its economic classifications 
align with SNA93 and other international standards to ensure comparability across its own 
economic collections and outputs as well as with those of other statistical organisations. 


Accuracy and Reliability 


For most users, accuracy and reliability are among the most sought after attributes of data. 
Accuracy can be defined as the proximity of an estimate to some notional true value. The true 
value is considered notional because, in practice, most statistics cannot be measured with 
perfect accuracy. Reliability is the proximity of initial and intermediate estimates for a particular 
period to the ‘final’ estimate for that period. In practice, at least in the Australian national 
accounts, accuracy and reliability tend to be interwoven and reinforcing. Ideally, as the estimate 
for a particular period passes through a sequence of revisions the size of the revisions gets 
smaller (the statistics become more reliable) and the estimate moves closer to the true value (the 
statistics become more accurate), but in practice this is sometimes not the case. By being aware 
of the factors that influence accuracy, judgements can be made as to the extent of error likely to 
be associated with an estimate. 


For assessing the accuracy and reliability of the Australian national accounts users can consider 
two information sources. First, 5216.0 provides a detailed account of the sources and methods 
underlying the data compiled as well as an overall assessment of data quality for each of the key 
components of GDP. Second, periodically, the ABS undertakes an analysis of the revisions that it 
makes to estimates of GDP and its components. Such an analysis was released in an article 
published in the July 1998 edition of Australian Economic Indicators (cat.no.1350.0). Another 
analysis is planned in the next 12 months. A recent study by staff at the Reserve Bank of 
Australia showed that although revisions to quarterly GDP growth rates have been significant in 
some quarters the pattern of revisions has shown a marked improvement in the last decade. It is 
our belief that this is due to the introduction of improved methods and data sources since 1990. 


Improvements in accuracy and reliability continue to be an aim. Currently, the ABS is embarking 
on a major program of implementing new ways of processing business statistics, new 
organisational arrangements, new methods and new technologies, all of which are expected to 
have a positive impact on the national accounts. Internal processes will also be improved through 
the application of quantifiable indicators of quality throughout the statistical processing cycle. 


Serviceability 


This dimension of quality relates to the need to ensure that: 
e data are produced and disseminated in a timely and regular manner 
e data are consistent both internally and with other related datasets 


e that a predictable revisions policy is used. 


Release practices: In releasing national accounts statistics, the ABS follows established 
principles that results should be made available as soon as practicable after the reference period, 
and should be available equally to all users. Australian national accounts statistics include major 
economic indicators that are in strong demand and can influence financial markets. Therefore, 
care is taken to ensure that no user receives the statistics before the designated release time, 
with a small number of exceptions. These exceptions relate to designated officers in certain 
government departments who are required to prepare briefing material on the statistics for their 
ministers; they are subject to a strict embargo until the official release of the national accounts. 
Importantly, the ABS announces the dates of its national accounts releases six months in 
advance and sticks to these announced dates in all but the most exceptional circumstances. 


Revisions policy: It is ABS policy to inform users of any significant revisions, and where 
appropriate to revise past time series and advise users accordingly. A comprehensive revisions 
policy for national accounts statistics can be obtained on request. 


Timeliness: The timeliness of data releases is measured by the gap between the reference 
period and the date of publication of results. There is an important trade-off between accuracy 
and detail on the one hand and timeliness of the release of statistics on the other. The source 
data that are used to compile the national accounts are available with varying degrees of 
timeliness, frequency, accuracy and detail. The data sources providing more detailed and 
accurate data tend to be those that are less frequent and/or less timely. Because of this, within a 
given level of resources, improvements in timeliness can generally only be made at the expense 
of accuracy and detail. 


The trade-off between timeliness, and accuracy and detail, is accommodated by way of the 
sequence of releases of national accounts publications. The first published quarterly estimates, 
which have considerable detail, are usually made available about 65 days after the reference 
period. The detailed annual estimates are released between 4 and 5 months after the reference 
period. The most detailed national accounts estimates relating to production, income and 
expenditure are those contained in the input-output tables which typically become available about 
3.5 years after the reference period. Australia's national accounts rate well against those of other 
countries in terms of their timeliness - particularly when the level of detail made available is taken 
into consideration. 


Relevance: The ABS recognises that, in order to be relevant to informed decision making, 
debate and discussions, its national accounts statistics must be timely and relatable to other 
data. To support this they are placed in an appropriate statistical framework. Apart from the 
extensive range of existing data, the ABS continues to monitor and anticipate users needs. 
Ensuring ABS national accounts outputs support decision-making is also achieved by extensive 
consultation with state and territory governments and other program-specific user groups. The 
national accounts program proposes to increase its range of statistical output by expanding the 
range of data released and increasing the range of electronic releases. 


Training and assistance: Part of the ABS vision is to provide leadership by assisting decision 
makers to better understand how to access, use and interpret national accounts information to 
improve overall decision making or research activities. In line with this objective, the ABS 


regularly provides training sessions within Australia covering the concepts underlying national 
accounts data and the sources and methods used in their derivation. In addition, statistical advice 
and technical assistance is regularly provided to national accountants in other countries. 


In future, the ABS also plans to further expand its service to users through the provision of 
courses covering the various dimensions of quality and how knowledge of quality should inform 
the use of statistics. It is hoped that, in turn, this will encourage users of to provide feedback to 
the ABS about areas of its statistical program which they feel need further improvement. 


Comparability: For the most part, SNA93 (in conjunction with related international standards, 
such as the IMF's Balance of Payments Manual) is used as the framework for all ABS economic 
statistics, not just the national accounts. The use of this common framework helps to achieve 
comparability between the national accounts and other ABS economic statistics. In addition, the 
ABS has a range of standard economic classifications including those for institutional sector, 
industry and commodity (product). These classifications are used in the national accounts and in 
other ABS statistics and, occasionally, in non-ABS statistics. 


Another way in which the comparability of the national accounts with other ABS statistics is 
achieved is through the use of a common statistical units model. Statistics (particularly those 
pertaining to industry) from two sources are more comparable when both sources use the same 
statistical unit. However, for a number of reasons it is not always possible to use the same type of 
statistical unit in all collections and thus the ABS's units model provides for a number of different, 
but relatable, types of statistical units. Statistical comparability is also enhanced when the 
populations for statistical collections are obtained from a common source; the ABS's Business 
Register is such a source for the ABS economic collections. 


ABS national accounts are likely to be more compatible with other ABS statistics than with non- 
ABS statistics. Accordingly, when the national accounts are compared with non-ABS statistics, 

users need to be aware of any conceptual or methodological differences that may impact upon 
comparability. 


Accessibility 


This quality dimension relates to the need to ensure that clearly presented statistics are readily 
available. It also relates to the need for clear information on how the data estimates are derived 
and how to interpret them correctly, including the need for prompt and knowledgeable support to 
users of the statistics from the data producers. 


The most well-known national accounts outputs are the quarterly publication, Australian 
National Accounts: National Income, Expenditure and Product (cat.no.5206.0) and the 
annual publications, Australian System of National Accounts (cat.no.5204.0) and Australian 
National Accounts: State Accounts (cat.no.5220.0). Other national accounts releases include 
the Australian National Accounts: Financial Accounts (cat.no.5232.0), Australian National 
Accounts: Input Output Tables (cat. no. 5209.0), the closely related Australian National 
Accounts: Input Output Tables (Commodity Details) (cat.no.5215.0) and the Australian 
National Accounts: Tourism Satellite Account (cat.no.5249.0). 


In order to make national accounts statistics readily accessible, they are usually released ina 
number of formats. All of the outputs are released as traditional paper publications, which are 
available by mail and ‘over the counter’ in each capital city, and copies of the key national 
accounts publications are provided to public libraries around Australia. 


The paper publications are supplemented by a variety of electronic releases, including AusStats 
and tailored time-series releases, for catalogue numbers 5206.0, 5204.0 and 5220.0. Increasing 
emphasis on electronic dissemination is accepted as a strategic direction for the release of ABS 


data. Accordingly, the national accounts program has considerably expanded its electronic 
release of data over the past year. Currently, 118 tables are released electronically each quarter 
for 5206.0, many containing data back to 1959-60. Annually, 103 tables are released for 5204.0 
and 37 tables for 5220.0. These electronic spreadsheets are available on subscription but can 
also be purchased on-line through the ABS web site. 


The main features section of each national accounts publication is available on the ABS web site 
free of charge. The ABS also makes its national accounts statistics available to secondary 
providers for inclusion in their products. National accounts statistics are also provided to the 
media, who in turn provide extensive coverage of the statistics. Users requiring more detail than 
that provided in the standard national accounts releases can seek information on request and, 
subject to cost-recovery charges, the information will be provided if it is available. Information 
about the full range of ABS national accounts publications and products is provided in the 
Catalogue of Publications and Products (cat.no.1101.0) or the ABS web site. 


Information about the national accounts 


The ABS, like other statistical offices, has an extensive range of ways to inform users about its 
published national accounts data. The concepts, sources and methods publication outlines the 
major concepts and definitions, describes the data sources and methods used to prepare the 
estimates and provides information about the ways in which the relevant international concepts 
and standards have been implemented in Australia. It also discusses the accuracy and reliability 
of national accounting concepts and estimates. It is a reference document suitable for data users 
requiring an in-depth understanding of data concepts and quality. The publication is available 
from the ABS's web site or on CD-ROM. Supplementing and updating this publication are an 
assortment of information, working papers and feature articles which are used to draw attention 
to issues impacting on the data such as changes to the classifications, systems, concepts or 
standards, major data revisions and changes in data dissemination practices. 


Significant sources of information about data quality are the annual and quarterly publications. 
Paper publications include text that describes their content, accuracy and the production process. 
Such information is necessary for users to interpret the data and make sensible judgments about 
their relevance, coherence and accuracy. The front pages contain key figures and key points 
which draw attention to the most relevant data. Information is included about future releases and 
about any changes in this issue. Main features describe the major elements of the data, the 
tables and graphs contain footnotes to alert users to revisions and other relevant issues that 
impact the data series. The final pages of both the quarterly and annual publications contain 
‘Explanatory Notes’ and a 'Glossary' which describe in detail scope and coverage, data item 
definitions and classifications, chain volume measures and revisions policy. In accordance with 
ABS policy, each publication also explicitly names a person who can be approached for further 
information. 


Future directions 


The ABS is strongly committed to maintaining and improving the quality of all aspects of its 
statistical program, including the provision of enhanced information on the quality of its statistics. 
Involvement in the development of international quality frameworks will continue, in particular for 
those being developed by the IMF for macro-economic statistics. The ABS will also continue to 
regularly review its processes and procedures to ensure their effectiveness and reliability and to 
ensure that the adoption of 'best practice’ continues across all its statistical programs. 


While significant amounts of information about the different dimensions of national accounts data 
quality are already available, including concepts, sources and methods publications, user guides 
and the explanatory notes in publications, it is comparatively more difficult to obtain a short 
qualitative summary of all this information, especially electronically. Additionally, much of the 


information collected about data quality is only referenced internally as part of the continuous 
improvement process. 


The challenge in the future will be to integrate the information that is available in internal systems 
into public quality declarations. The ABS is proposing to develop Quality Summaries for each of 
its statistical programs that will be concise enough to be useful to the user. Through the use of 
explicit links between these external perceptions of data quality and the internal processes which 
generate statistical information, the proposed quality summaries will also be able to identify 
quality gains from the implementation of new procedures and processes as well as identify areas 
requiring further improvement. Summaries will be made available on the internet via the ABS 
web site, and will be automatically supplied with purchased tables, as well as being incorporated 
into existing explanatory notes within ABS publications. 


Further advances in electronic dissemination will offer additional opportunities to develop and 
enhance presentation methods. Finally, it will remain a priority of the national accounts program 
to continue to provide a high quality service to its users by keeping them informed of statistical 
developments that impact on the national accounts through the use of information papers, 
seminars and special articles in publications. 


Further information 
Chapter 20 of 5216.0 contains further information about the quality of Australia's national 


accounts. For further information on the national accounts quality program contact Margaret 
Hausknecht on (02)62526250 or by email margaret.hausknecht@abs.gov.au. 
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